Expectation Conditional Maximization Research Articles

Constrained inference in scaled mixed effects models with applications to London Junior School Project data

Multiple outcomes arise frequently in many different settings. Mixed effect models are a useful tool for the analyses of such data. However, when outcomes are measured on different scales, analyses based on any one scale are misleading. Often parameters of the model are subject to known order restrictions. To incorporate heterogeneity across different outcomes, we propose a scaled linear mixed effect model. To estimate parameters, we propose a maximum likelihood estimation procedure based on a restricted version of the expectation-conditional maximization either algorithm. Constrained hypotheses testing procedures are developed using likelihood ratio tests. The empirical significance levels and powers are studied using simulations. This article shows that incorporating the restrictions improves the mean squared errors of the estimates and the power of the tests. The methodology is applied on the London Junior School Project data incorporating known restrictions of patterns of scores. Journal of Statistical Research 2023, Vol 57, No.1-2, pp.81-94

Time series clustering based on latent volatility mixture modeling with applications in finance

Modeling financial time series data poses a significant challenge in the realm of time series analysis. The Autoregressive Conditional Heteroskedasticity (ARCH) model stands out as a potent tool for capturing time-varying volatility and heteroskedasticity in financial data. However, conventional ARCH models display sensitivity to departures from normality, leading to the development of extensions employing more flexible distributions. In this context, we propose a robust enhancement to the mixture of ARCH (MoARCH) model by integrating normal mean–variance mixture (NMVM) distributions to model component errors. The stochastic representation of the proposed model allows for a straightforward implementation of an Expectation Conditional Maximization Either (ECME) algorithm for obtaining Maximum Penalized Likelihood estimates (MPL). To thoroughly evaluate the model, we conduct four simulation studies to explore finite-sample properties, assess MPL estimators, scrutinize model robustness, and evaluate the accuracy of our proposal in fitting, clustering, and forecasting. Practical applications further highlight the effectiveness of our methodology, showcasing successful implementations across diverse real datasets.

A novel mixture model for characterizing human aiming performance data

Fitts’ law is often employed as a predictive model for human movement, especially in the field of human-computer interaction. Models with an assumed Gaussian error structure are usually adequate when applied to data collected from controlled studies. However, observational data (often referred to as data gathered ‘in the wild’) typically display noticeable positive skewness relative to a mean trend as users do not routinely try to minimize their task completion time. As such, the exponentially modified Gaussian (EMG) regression model has been applied to aimed movements data. However, it is also of interest to reasonably characterize those regions where a user likely was not trying to minimize their task completion time. In this article, we propose a novel model with a two-component mixture structure—one Gaussian and one exponential—on the errors to identify such a region. An expectation-conditional-maximization (ECM) algorithm is developed for estimation of such a model and some properties of the algorithm are established. The efficacy of the proposed model, as well as its ability to inform model-based clustering, are addressed in this work through extensive simulations and an insightful analysis of a human aiming performance study.

Estimation for vector autoregressive model under multivariate skew-t-normal innovations

Current procedures for estimating the parameters of [Formula: see text]th order vector autoregressive (VAR [Formula: see text]) model are usually based on assuming that the ensuing error distribution is multivariate normal. But there exists large body of evidence that several data encountered in real life are skewed; thereby making estimators derived based on normality assumption not suitable in such scenarios. This prompts for the search of appropriate methods for skewed distributions. Therefore, this article proposes estimators for the mean and covariance matrices of the [Formula: see text] model under multivariate skew- [Formula: see text]-normal (MSTN) distribution. Also, estimators for the shape and skewness parameters are provided. The expectation conditional maximization (ECM) and its extension the expectation conditional maximization either (ECME) algorithms are the tools used to derive the estimators. The performance of the estimators were examined through extensive simulations, and results show that they compete favourably with other numerical methods especially when the underlying distribution is skewed. The usefulness of our estimators was illustrated using a real data set on some US economic indicators. The VAR [Formula: see text] model under MSTN distribution provides a good fit, better than [Formula: see text] model under the assumption of normality.

The generalized scale mixtures of asymmetric generalized normal distributions with application to stock data

<abstract><p>In this paper, we introduced a family of distributions with a very flexible shape named generalized scale mixtures of generalized asymmetric normal distributions (GSMAGN). We investigated the main properties of the new family including moments, skewness, kurtosis coefficients and order statistics. A variant of the expectation maximization (EM)-type algorithm was established by combining the proflie likihood approach (PLA) with the classical expectation conditional maximization (ECM) algorithm for parameter estimation of this model. This approach with analytical expressions in the E-step and tractable M-step can greatly improve the computational speed and efficiency of the algorithm. The performance of the proposed algorithm was assessed by some simulation studies. The feasibility of the proposed methodology was illustrated through two real datasets.</p></abstract>

Mixtures of factor analysers with censored responses and external covariates: An application to educational data from Peru.

Analysing data from educational tests allows governments to make decisions for improving the quality of life of individuals in a society. One of the key responsibilities of statisticians is to develop models that provide decision-makers with pertinent information about the latent process that educational tests seek to represent. Mixtures of factor analysers (MtFA) have emerged as a powerful device for model-based clustering and classification of high-dimensional data containing one or several groups of observations with fatter tails or anomalous outliers. This paper considers an extension of MtFA for robust clustering of censored data, referred to as the MtFAC model, by incorporating external covariates. The enhanced flexibility of including covariates in MtFAC enables cluster-specific multivariate regression analysis of dependent variables with censored responses arising from upper and/or lower detection limits of experimental equipment. An alternating expectation conditional maximization (AECM) algorithm is developed for maximum likelihood estimation of the proposed model. Two simulation experiments are conducted to examine the effectiveness of the techniques presented. Furthermore, the proposed methodology is applied to Peruvian data from the 2007 Early Grade Reading Assessment, and the results obtained from the analysis provide new insights regarding the reading skills of Peruvian students.

Parameter estimation of the multivariate unrestricted skew-normal distribution using ECM algorithm

The probability density function of the multivariate unrestricted skew-normal (SUN) distribution, corresponding to a screened normal density, allow to modeling skewness and kurtosis in data in terms of a skewness parameter vector and a truncation parameter matrix. These parameters are related to the shape and heavy-tails of the density. In this article, we present the Expectation/Conditional Maximization (ECM) algorithm for the SUN distribution based on a hierarchical stochastic representation. In addition, behavior of ECM algorithm’s steps is measured using an information theoretic approach based on Jeffrey’s divergence and related homogeneity test. Usefulness of the proposed method is illustrated by an application to Chilean economic perception data.

Stochastic Maximum Likelihood Direction Finding in the Presence of Nonuniform Noise Fields

The maximum likelihood (ML) technique plays an important role in direction-of-arrival (DOA) estimation. In this paper, we employ and design the expectation–conditional maximization either (ECME) algorithm, a generalization of the expectation–maximization algorithm, for solving the ML direction finding problem of stochastic sources, which may be correlated, in unknown nonuniform noise. Unlike alternating maximization, the ECME algorithm updates both the source and noise covariance matrix estimates by explicit formulas, and can guarantee that both estimates are positive semi-definite and definite, respectively. Thus, the ECME algorithm is computationally efficient and operationally stable. Simulation results confirm that the ECME algorithm can efficiently obtain the ML based DOA estimate of each stochastic source.

Multivariate Contaminated Normal Censored Regression Model: Properties and Maximum Likelihood Inference

The Multivariate Contaminated Normal (MCN) distribution which contains two extra parameters with respect to parameters of the multivariate normal distribution, one for controlling the proportion of mild outliers and the other for specifying the degree of contamination, has been widely applied in robust statistics in the case of elliptically heavy-tailed empirical distributions. This article extends the MCN model to data with possibly censored values due to limits of quantification, referred to as the MCN with censoring (MCN-C) model, and further establishes the censored multivariate linear regression model where the random errors have the MCN distribution, named as the MCN censored regression (MCN-CR) model. Two computationally feasible Expectation Conditional Maximization (ECM) algorithms are developed for maximum likelihood estimation of MCN-C and MCN-CR models. An information-based method is used to approximate the standard errors of location parameters and regression coefficients. The capability and effectiveness of the MCN-C and MCN-CR models are illustrated via two real-data examples. A simulation study is conducted to investigate the superiority of the proposed models in terms of fit, accuracy of parameter estimation and censored data recovery as compared with classical approaches. Supplementary materials for this article are available online.

An expectation conditional maximization algorithm for the skew-normal based stochastic frontier model

An expectation conditional maximization algorithm for the skew-normal based stochastic frontier model

Analyzing COVID-19 data in the Canadian province of Manitoba: A new approach

The basic homogeneous SEIR (susceptible–exposed–infected–removed) model is a commonly used compartmental model for analysing infectious diseases such as influenza and COVID-19. However, in the homogeneous SEIR model, it is assumed that the population of study is homogeneous and, one cannot incorporate individual-level information (e.g., location of infected people, distance between susceptible and infected individuals, vaccination status) which may be important in predicting new disease cases. Recently, a geographically-dependent individual-level model (GD-ILM) within an SEIR framework was developed for when both regional and individual-level spatial data are available. In this paper, we propose to use an SEIR GD-ILM for each health region of Manitoba (central Canadian province) population to analyse the COVID-19 data. As different health regions of the population under study may act differently, we assume that each health region has its own corresponding parameters determined by a homogeneous SEIR model (such as contact rate, latent period, infectious period). A Monte Carlo Expectation Conditional Maximization (MCECM) algorithm is used for inference. Using estimated parameters we predict the infection rate at each health region of Manitoba over time to identify highly risk local geographical areas. Performance of the proposed approach is also evaluated through simulation studies.

Sparse and smooth functional data clustering

A new model-based procedure is developed for sparse clustering of functional data that aims to classify a sample of curves into homogeneous groups while jointly detecting the most informative portions of the domain. The proposed method is referred to as sparse and smooth functional clustering (SaS-Funclust) and relies on a general functional Gaussian mixture model whose parameters are estimated by maximizing a log-likelihood function penalized with a functional adaptive pairwise fusion penalty and a roughness penalty. The former allows identifying the noninformative portion of the domain by shrinking the means of separated clusters to some common values, whereas the latter improves the interpretability by imposing some degree of smoothing to the estimated cluster means. The model is estimated via an expectation-conditional maximization algorithm paired with a cross-validation procedure. Through a Monte Carlo simulation study, the SaS-Funclust method is shown to outperform other methods that already appeared in the literature, both in terms of clustering performance and interpretability. Finally, three real-data examples are presented to demonstrate the favourable performance of the proposed method. The SaS-Funclust method is implemented in the R package sasfunclust, available on CRAN.

Flexible modeling of multiple nonlinear longitudinal trajectories with censored and non-ignorable missing outcomes.

Multivariate nonlinear mixed-effects models (MNLMMs) have become a promising tool for analyzing multi-outcome longitudinal data following nonlinear trajectory patterns. However, such a classical analysis can be challenging due to censorship induced by detection limits of the quantification assay or non-response occurring when participants missed scheduled visits intermittently or discontinued participation. This article proposes an extension of the MNLMM approach, called the MNLMM-CM, by taking the censored and non-ignorable missing responses into account simultaneously. The non-ignorable missingness is described by the selection-modeling factorization to tackle the missing not at random mechanism. A Monte Carlo expectation conditional maximization algorithm coupled with the first-order Taylor approximation is developed for parameter estimation. The techniques for the calculation of standard errors of fixed effects, estimation of unobservable random effects, imputation of censored and missing responses and prediction of future values are also provided. The proposed methodology is motivated and illustrated by the analysis of a clinical HIV/AIDS dataset with censored RNA viral loads and the presence of missing CD4 and CD8 cell counts. The superiority of our method on the provision of more adequate estimation is validated by a simulation study.

An Improved Measurement Error Model for Analyzing Unreplicated Method Comparison Data under Asymmetric Heavy-Tailed Distributions

Method comparison studies mainly focus on determining if the two methods of measuring a continuous variable are agreeable enough to be used interchangeably. Typically, a standard mixed-effects model uses to model the method comparison data that assume normality for both random effects and errors. However, these assumptions are frequently violated in practice due to the skewness and heavy tails. In particular, the biases of the methods may vary with the extent of measurement. Thus, we propose a methodology for method comparison data to deal with these issues in the context of the measurement error model (MEM) that assumes a skew- t (ST) distribution for the true covariates and centered Student’s t (cT) distribution for the errors with known error variances, named STcT-MEM. An expectation conditional maximization (ECM) algorithm is used to compute the maximum likelihood (ML) estimates. The simulation study is performed to validate the proposed methodology. This methodology is illustrated by analyzing gold particle data and then compared with the standard measurement error model (SMEM). The likelihood ratio (LR) test is used to identify the most appropriate model among the above models. In addition, the total deviation index (TDI) and concordance correlation coefficient (CCC) were used to check the agreement between the methods. The findings suggest that our proposed framework for analyzing unreplicated method comparison data with asymmetry and heavy tails works effectively for modest and large samples.

Multivariate measurement error models with normal mean‐variance mixture distributions

The class of normal mean‐variance mixture (NMVM) distributions is a rich family of asymmetric and heavy‐tailed distributions and has been widely considered in parametric modeling of the data for robust statistical inference. This paper proposes an extension of measurement error models by assuming the NMVM distributions for the unobserved covariates and error terms in the model, referred to as the NMVM‐MEM. An expectation conditional maximization either (ECME) algorithm is developed to compute the maximum likelihood (ML) estimates of model parameters. Furthermore, an information‐based approach is performed to derive the asymptotic covariance matrix of ML estimators. The analysis of a blood pressure dataset illustrates the superiority of NMVM‐MEM to accommodate asymmetry and outliers over the normal counterpart. Two simulation studies are undertaken to validate our proposed techniques.

Robust mixture regression modeling based on the normal mean-variance mixture distributions

Mixture regression models (MRMs) are widely used to capture the heterogeneity of relationships between the response variable and one or more predictors coming from several non-homogeneous groups. Since the conventional MRMs are quite sensitive to departures from normality caused by extra skewness and possible heavy tails, various extensions built on more flexible distributions have been put forward in the last decade. The class of normal mean-variance mixture (NMVM) distributions that arise from scaling both the mean and variance of a normal random variable with a common mixing distribution encompasses many prominent (symmetric or asymmetrical) distributions as special cases. A unified approach to robustifying MRMs is proposed by considering the class of NMVM distributions for component errors. An expectation conditional maximization either (ECME) algorithm, which incorporates membership indicators and the latent scaling variables as the missing data, is developed for carrying out maximum likelihood (ML) estimation of model parameters. Four simulation studies are conducted to examine the finite-sample property of ML estimators and the robustness of the proposed model against outliers for contaminated and noisy data. The usefulness and superiority of our methodology are demonstrated through applications to two real datasets.

Extending the Fellegi-Sunter record linkage model for mixed-type data with application to the French national health data system

Probabilistic record linkage is a process of combining data from different sources, when such data refer to common entities and identifying information is not available. A probabilistic record linkage framework that takes into account multiple non-identifying information that this is limited to simple binary comparison between matching variables has been previously proposed. An extension of this method is proposed for mixed-type comparison vectors. A mixture model for handling comparison values of low prevalence categorical matching variables, and a mixture of hurdle gamma distribution for handling comparison values of continuous matching variables have been developed. The parameters are estimated by means of the Expectation Conditional Maximization (ECM) algorithm. Through a Monte Carlo simulation study, both the posterior probability estimation for a record pair to be a match and the prediction of matched record pairs are evaluated. The simulation results indicate that the proposed methods outperform existing ones in most considered cases. The proposed methods are applied on a real dataset, to perform linkage between a registry of patients suffering from venous thromboembolism in the Brest district area (GETBO) and the French national health information system (SNDS).

Maneuvering extended object tracking based on constrained expectation maximization

This paper addresses the problem of maneuvering extended object tracking, in which the object extension (OE) and the turn rate are identified simultaneously. Due to its non-linearity with respect to the turn rate, the state transition is converted into a linear form of a newly defined hyper-parametric vector by parameter substitution. For the hyper-parametric equality constraints (HECs) introduced in the transformation, a constrained expectation conditional maximization algorithm is designed. The HECs are projected onto the conditional expectation function for regularization, so as to realize the iterative identification of multi-parameter (i.e., OE and turn rate). The transformation and CECM optimization bring the advantage of avoiding nonlinear model approximation, which is important for the convergence and accuracy of state estimation. Finally, simulation results demonstrate the superiority of the proposed method in terms of both estimation accuracy and identification effectiveness.

Adaptive Radar Detection in the Presence of Missing-data

This article deals with the problem of adaptive radar detection in a missing-data context, where the complete observations (i.e., downstream information loss mechanisms) are characterized by homogeneous Gaussian disturbance with an unknown but possibly structured covariance matrix. The detection problem, formulated as a composite hypothesis test, is tackled by resorting to suboptimal design strategies, leveraging the generalized likelihood ratio criterion demanding appropriate maximum likelihood estimates (MLEs) of the unknowns under both hypotheses. Capitalizing on some possible <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> knowledge about the interference covariance matrix structure, the optimization problems involved in the MLE computation are handled by employing the expectation–maximization (EM) algorithm or its expectation–conditional maximization and multicycle EM variants. At the analysis stage, the performance of the devised architectures is assessed both via Monte Carlo simulations and on measured data for some covariance matrix structures of practical interest.

Adaptive estimation of autoregression models under long-tailed symmetric distribution

Non-normal innovations in autoregression models frequently occur in practice. In this situation, least squares (LS) estimators are known to be inefficient and non-robust, and maximum likelihood (ML) estimators need to be solved numerically, which becomes a daunting task. In the literature, the modified maximum likelihood (MML) estimation technique has been proposed to obtain the estimators of model parameters. While an explicit solution can be found via this method, the requirement of knowing the shape parameter becomes a drawback, especially in machine learning. In this study, we use the adaptive modified maximum likelihood (AMML) methodology, which combines the MML with Huber’s M-estimation so that this assumption is relaxed. The performance of the method in terms of efficiency and robustness is analyzed via simulation and compared to LS, MML and ML estimates that are obtained numerically via the Expectation Conditional Maximization (ECM) algorithm. Test statistics are proposed for the crucial parameters of the model. The results show that the AMML estimators are preferable in most of the settings according to the mean squared error (MSE) criterion and the test statistics based on AMML method are more robust than the others. Furthermore, both real life and synthetic data examples are given.